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SUPPLEMENTARY  NOTES 


ABSTRACT 


The  output  of  an  industrial  plant  (such  as  a  chemical  plant)  will  in  general 
depend  on  "plant  conditions"  i.e.,  factors  characterizing  the  process  such  as 
the  temperatures  at  which  stages  of  the  process  are  run,  the  concentrations  at 
which  chemicals  are  applied,  the  concentrations  at  which  catalysts  are  used,  etc. 

In  order  to  "improve"  such  a  process,  it  is  desirable  to  estimate  the  functional 
dependence  of  the  target  output  y  on  the  plant  conditions  denoted  by  Xj,  x9,...,x . 
For  this  purpose  is  it  customary  to  use  data  from  a  planned  experiment  usually  n 
using  a  "pilot  plant"  in  which  the  plant  conditions  X£  are  deliberately  determined 
by  the  experimentor  in  accordance  with  an  experimental  design.  In  such  situations 
the  plant  conditions  at  which  the  pilot  plant  is  run  are  often  "optimized''. 

The  criterion  of  optimization  here  used  is  the  "generalized  variance"  of  the 
estimated  coefficients  occuring  in  the  mathematical  law  representing  the  dependence 
of  the  output  on  the  inputs.  The  particular  type  of  experimental  design  considered 
in  this  technical  report  are,  however,  restricted  to  satisfy  a  well  established 
pattern  known  under  the  name  of  a  "Composite  Design"  and  the  parameters  describing 
such  a  composite  design  are  optimized  in  the  sense  of  minimizing  the  generalized 
variance. 

The  composite  designs  considered  are  both  of  a  symmetrical  and  asmetrical 
type  and  may  or  may  not  involve  the  use  of  fractional  factorials. 
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COMPOSITE  DESIGNS  AND  THEIR  OPTIMIZATION 
1.1  Introduction 

An  important  problem  in  industry  is  the  improvement  (which 
will  hopefully  lead  to  the  optimization)  of  a  process.  This  report 
attacks  a  problem  in  this  area. 

To  improve  an  "ongoing  process"  often  it  is  desired  to  examine 
the  relationship  between  variables  in  a  process,  say  between  a  y 
variable  when  y  represents  a  "response"  variable  and  x^,  •••»  xn> 

where  the  x's  represent  "input"  or  "control"  variables.  In  this 
report  we  develop  optimum  composite  designs  for  fitting  a  quadratic 
response  surface  under  the  standard  regression  assumptions,  and 
solve  some  related  design  of  experiments  problems.  The  optimization 
criterion  we  use  is  the  minimization  of  the  generalized  variance 
(the  |X'X|  criterion). 

To  clarify  the  concept  of  an  ongoing  process  we  may  appropriately 
think  of  a  production  process  in  a  chemical  plant  producing  a  target 
output  y  of  a  chemical.  It  is  commonplace  that  this  output  will  depend 
on  "plant  conditions",  i.e.,  factors  characterizing  the  process  such 
as  the  temperatures  at  which  stages  of  the  process  are  run,  the 
concentrations  at  which  chemicals  are  input,  the  concentrations  at 
which  catalysts  are  used,  etc.  In  order  to  "improve"  such  a  process, 
it  is  desirable  to  estimate  the  functional  dependence  of  the  target 
output  y  on  the  plant  conditions  denoted  by  x^,  X£,  . ..,  xfl.  The  data 
available  for  such  an  improvement  may  be  of  essentially  two  kinds: 

(a)  We  may  have  "plant  data"  in  which  the  target  output  y 
is  observed  along  with  measured  records  of  the  plant 
conditions  x^,  X2*  •••*  xn»  or 

(b)  We  may  have  data  from  planned  experiments  usually  us 
using  a  "pilot  plant"  in  which  the  plant  conditions 
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are  deliberately  determined  by  the  experimenter 
in  accordance  with  an  experimental  plan,  after  which 
the  associated  target  output  y  of  the  pilot  plant  is 
measured. 

The  concept  of  using  an  experimental  design  to  estimate  the 
mathematical  relationship  between  the  target  output  and  plant  condi¬ 
tions  is  normally  restricted  to  case  (b)  above. 

This  report  will  be  exclusively  concerned  with  case  (b).  More 
specifically,  this  report  is  concerned  with  choosing  the  set  of 
plant  conditions,  x^,  at  which  the  pilot  plant  is  to  be  run, 
called  the  experimental  design, in  such  a  manner  that  certain 
features  of  the  mathematical  relationship  can  be  estimated  with 
"optimum”  precision.  We  shall  confine  our  study  to  a  situation  in 
which  the  mathematical  relationship  is  postulated  to  be  of  "second 
order",  i.e. ,  what  is  known  as  a  "quadratic  response  surface". 

It  will  be  assumed  that  the  expected  yield  of  the  pilot  plant  will 
be  given  by  such  a  response  surface  and  that  the  observed  target 
outputs  will  differ  from  their  expectations  by  independent  equal 
variance  residuals.  These  assumptions  are  the  standard  ones  in 
"regression  analysis".  Using  the  estimation  procedures  appropriate 
to  such  assumptions  will  result  in  estimates  of  the  so-called 
regression  coefficients,  i.e,,  effect  coefficients  of  the  x^, 
their  products,  and  squares.  An  estimate  of  the  individual 
variances  of  such  estimates  as  well  as  of  the  so-called  generalized 
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variance  of  the  coefficients  can  also  be  obtained.  The  magnitude 
of  this  generalized  variance  will  depend  on  the  experimental 
design  chosen.  The  objective  of  the  optimization  of  the  design 
will  be  the  minimization  of  this  generalized  variance. 

The  problem  of  the  optimization  of  experimental  designs  by 
the  minimization  of  the  generalized  variance  can  be  reduced  to 
a  mathematical  programming  problem  which  is  solved  on  a  high  speed 
computer.  This  approach  was  taken  by  Hartley  and  Ruud  [1969] 
and  by  Crowell  [1971].  The  disadvantage  of  this  attack  is  the 
complexity  of  the  programming  problem;  it  is  generally  a  non¬ 
linear  non-convex  programming  problem.  Thus  a  global  optimum 
cannot  be  guaranteed. 

By  restricting  our  attention  to  composite  designs  we  will 
be  able  to  avoid  formulating  a  programming  problem.  We  will 
attack  the  problem  directly  and  will  minimize  the  generalized 
variance  for  composite  designs  restricted  to  an  n  dimensional 
hypercube  where  n  is  the  number  of  variables  under  study.  The 
minimization  of  the  generalized  variance  is  equivalent  to  the 
maximization  of  |x*x|  where  X  is  a  N  x  p  "expanded"  design  matrix 
having  N  rows  and  p  columns,  one  column  for  each  coefficient  to 
be  estimated.  The  X^^  matrix  is  expanded  from  the  design  matrix 
Xp,  a  N  x  n  matrix,  which  has  one  column  for  each  variable  under 
study.  For  a  full  quadratic  model  p  *  (n+l)(n+2)/2  ■  1  +  n  +  n  + 
In  the  X  matrix  there  is  one  column  for  the  constant  term,  n  for 
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the  linear  terms,  n  for  the  quadratic  terms,  and  (£)  columns 
for  the  interaction  terms. 

Four  types  of  composite  designs  will  be  examined: 

(1)  A  "syr-metric"  composite  denign  (a  "symmetric”  composite 
desii;;x  I'&s  "suar"  point  distance  equal  to  +  a) . 

(2)  A  "symmetric"  smallest  composite  der^gn  (a  saturated 
design  for  which  w  =-  p  *  (n+1) (n+2)/2). 

(3)  An  unsymraevvio  composite  design  (an  unsymmetric  composite 
design  has  "star"  point  distance  equal  to  (+1,  -a)). 

(4)  An  unsymmetric  smallest  composite  design. 

Composite  designs  are  used  primarily  in  "response  surface 
analysis".  Box  and  Wilscr’s  [1951]  paper  opened  this  important 
and  practical  field,  and  Box  and  Draper’s  [1959]  paper  discussed 
some  optimization  problems  in  this  area.  Hartley  and  Ruud  [1969] 
and  Crowell  [1971]  give  a  good  introduction  to  the  field  of 
optimal  experimental  designs  and  contain  additional  references, 

1,2  Optimization  of  Symmetric  Composite  Designs 
1.2.1  Alias  structure  I  »  ABCDE 

A  symmetric  composite  in  n  variables  which  is  used  to  estimate 
all  terms  in  a  quadratic  response  surface  consists  of: 
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lr  n 

(1)  A  proper  fraction  (say  a  1/2  fraction)  of  a  2 
factorial  array.  This  array  of  2  points  will  be 
placed  on  a  hypercube  having  sides  of  length  two 
centered  at  (0,  0,  ...,  0)lxn« 

(2)  r  center  points. 

(3)  Two  "star"  points,  one  at  +a  and  one  ut  -ct,  for  each 
variable;  2n  points  in  all. 

Thus,  the  design  consists  of  N  »  2  +  2n  +  r  points  to  estimate 

p  ■  (n+l)(n+2)/2  terms.  Table  1.1  lists  the  composite  designs 
for  n  <_  11. 

We  are  now  ready  to  state  and  prove  Theorem  1.2. 

Theorem  1  *j  For  a  "symmetric”  composite  design  the  generalized 
variance  decreases  with  increasing  a,  where  a  is  the  star  point 
distance.  In  this  section  we  prove  the  result  for  alias  structure 
such  that  two  factor  interactions  are  aliased  with  three  or  higher 
order  interactions  (I  =  ABCDE),  and  in  the  next  section  we 
indicate  the  changes  necessitated  by  other  alias  struct«.res. 

Method  of  Proof: 

Step  1.  Indicate  the  structure  of  the  X^Xp  matrix. 

Step  2,  Write  out  the  X'X  „  matrix. 

pxp 

Step  3.  Solve  for  |x'x|  and  show  |X*X|  is  an  increasing 


function  of  a 


Table  1,1 


Common  Symmetric  Composite  Designs 


Variables 

Terms  in 
Quadratic 
Model 

Points  in 

Design 

(With  one  center  point) 

Star  and 
Center 
Point 

Fractional 

Factorial 

2 

6 

9 

5 

22  =  4 

3 

10 

15 

7 

23  *  8 

*  3 

10 

11 

7 

2 3-1  =  4 

4 

15 

25 

9 

24  =  16 

*  4 

15 

17 

9 

24-1  -  8 

5 

21 

43 

11 

25  =  32 

5 

21 

27 

11 

2 5-1  =  16 

6 

28 

77 

13 

26  «  64 

6 

28 

45 

13 

26"1  =  32 

*  6 

28 

29 

13 

26“2  =  16 

7 

36 

143 

15 

27  =  128 

7 

36 

79 

15 

2 7-1  *  64 

*  7 

36 

47 

15 

27"2  *  32 

8 

45 

273 

17 

28  »  256 

8 

45 

145 

17 

28-1  «  128 

8 

45 

81 

17 

2  *  64 

9 

55 

531 

19 

q 

2'  =  512 

9 

55 

275 

19 

9_1 

2^  1  ®  256 

7 


Table  1,1  (Continued) 

Common  Symmetric  Composite  Designs 


Variables 

Terms  in 
Quadratic 
Model 

Points  in 

Design 

(With  one  center  point) 

Star  and 
Center 
Point 

Fractional 

Factorial 

9 

55 

147 

19 

2 9-2  =  128 

*  9 

55 

83 

19 

29“3  =  64 

10 

66 

1045 

21 

210  =  1024 

10 

66 

533 

21 

210_1  =  512 

10 

66 

277 

21 

210**2  «  256 

10 

66  • 

149 

21 

210”3  «  128 

11 

78 

2071 

23 

211  *  2048 

11 

78 

1047 

23 

211"1  »  1024 

11 

78 

535 

23 

211"2  *  512 

11 

78 

279 

23 

211'3  =  256 

11 

78 

151 

23 

2 11-4  *=  128 

The  starred  designs  vlll  have  some  three  character  words  in  an 
alias  set  (I  «  ABC).  For  all  other  designs  it  is  possible  to 
pick  the  alias  structure  so  that  all  words  in  an  alias  set  have 
at  least  five  characters  (I  »  ABCDE), 
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Step  3.  The  X’X  matrix  la  a  diagonal  matrix  except  for  an 
pxp 

nxn  matrix  which  has  the  structure 


[al  +  bJJ’]nXn  *  [M] 


where 


a  ■  2(a^-c)^  +  2c^  +  4c(a^~c)  *  2a  ^ 


b  *  2n”*^(l-c)^  +  (r+2n-4)  c"  -  4c(ot^~c) 


I  is  an  nxn  identity  matrix,  and 


J  is  a  n  vector  of  ones  • 


(n) 

-.n-lc,o_2xn,on-fc\  2  i 


|x'x|  *  (2n’K+2ttfr)  (2n"*+2c»  r(2n"*)  z  |m|  , 


We  show  in  the  Appendix,  Theorem  [4],  that 


|m|  ■  an  +  nan~*b  *  a11  ^(a+nb)  , 


Substituting  a,  b,  and  c  in  |m|  we  find 


maamwrr  ™!«" r  '.nn-mrTOii-ji'.inB-.iii  FMiiaTcuiii"  "nniiran-' jti'h.  , ,  . . . . . . 1  "  ‘ ' 


12. 


Combining  fractions  using  the  last  two  terms  only,  we  obtain 
a  numerator  greater  than; 

2nf2a4  -  (4n-2)  a2  +  2n2  -  n  -  1] 

The  discriminant  of  the  term  in  brackets  is  4(-2n+3),  Thus 
for  n>  1,  d  log  |x*x|/da  is  nowhere  negative,  and  |x*x|  is 
increasing  with  a,  n«l  is  an  easy  special  case  for  which  it  is 
trivial  to  show  that  |X'x|  is  increasing  with  o  so  the  result 
holds  for  all  n. 

The  preceding  shows  that  the  optimal  symmetric  composite 
design  on  a  hypercube  with  sides  of  length  two  centered  at  the 
origin  has  a  *  1,0, 

In  Section  1.6.1  we  show  that  expanding  the  hypercube  by 
a  factor  k  Introduces  a  multiplying  factor  k2n^n+2^  to  the 
determinant.  A  £  increase  in  the  size  of  the  hypercube  gives 
a  multiplying  factor  of  (1  +  2n(n+2)  £)  to  the  determinant  when 
we  consider  only  first  order  terms.  For  an  £  increase  in  a 
from  1  to  l+£  a  multiplying  factor  for  the  determinant  of 
(1  +  6n£)  is  obtained.  Thus,  for  optimal  composite  designs  on 
a  hypercube  we  need  only  consider  fractional  factorials  with 
points  at  the  extremes  of  the  hypercube.  Similar  considerations 
hold  for  the  fractional  factorial  portion  of  all  designs  considered 
in  this  chapter.  The  above  result  can  be  found  in  Box  and 


Draper  [1971],  This  publication  cane  out  after  we  had  independently 
obtained  the  same  result.  They  do  not  note  that  this  form  of 
jx*x|  does  not  hold  for  all  alias  structures.  We  discuss  this 
in  the  next  section  of  our  paper. 


1.2.2  Other  alias  structures 


With  other  alias  structures  it  may  not  be  possible  to  estimate 
all  terms  in  a  quadratic  model.  The  estimable  terms  can  be  found 
using  Theorem  1  of  Hartley  [1959]  which  we  discuss  and  extend 
in  Section  1.6.  For  alias  structure  I  -  ABCD  the  proof  goes 
through  exactly  as  given  except  that  three  of  the  product  terms 
cannot  be  estimated. 

The  I  *  ABC  alias  structure  is  a  good  one  in  that  it  allows 
the  estimation  of  all  coefficients  of  a  quadratic  response  sv  -.ace. 
For  a  six  row,  six  column  minor  of  the  determinant  with  alias 
structure  I  *  ABC  we  obtain: 


2n~k+2a2 


2n"k+2a2 


2n~W 
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With  alias  structure  I  «  ABODE  the  off-diagonal  elements 

would  not  be  obtained.  For  this  minor  the  determinant  is 
n-k+1  2  3 

(2  o  )  ,  This  increasing  function  of  a  substitutes  for 

(2n~kf2a2)3  ^(n-k)  yith  alias  structure  I  ■  ABODE, 

Otherwise,  the  argument  is  unchanged. 

For  the  I  »  A  alias  structure  we  choose  a  subtraction  constant 

2 

b  to  make  the  X  column  orthogonal  to  the  X  column  and  note  that 
this  choice  of  b  makes  the  X^  column  orthogonal  to  every  column 
except  the  column  for  the  constant  terms. 

b  « 


2n"k(l-c)  +  2ct2  -  (2n+r)  c 


and  the  changed  minor  in  the  X'X  matrix  is 


u  X 

a 

u  2n~k  +  2n  +  r  2n“k  -  (2n"k+2n4r)b 

X  2n”k  -  (2n"k+2n+r)b  (2n"k+2n+r)b2  +  2n"k+1b  +  2n_k  +  2a2 

a 

This  minor  has  determinant 


2n"k(2ntr)  +  <2n“k+2n+r)  2«2 


an  increasing  function  of  o 


The  I  «  AB  alias  structure  is  not  recommended  for  n  >  2 
since  all  terms  in  a  quadratic  model  cannot  be  estimated,  I  «=  AB 
implies  AC  *  BC  and  only  one  of  the  pair  of  interactions  can  be 
included  in  the  model.  Two  minors  are  changed  by  this  alias 
structure.  They  ares 

u 

2°"k+2n+r 
2n'k 

with  determinants 

2n“k(2n+r)  and  2n+k+2  a2  +  4a4  , 

So  an  increasing  function  of  a  is  obtained, 

1.3  Optimization  of  Smallest  Composite  Designs 
Symmetric  in  the  Star  Points 

By  using  an  improper  fraction  of  a  2n  factorial  array  we  can 
estimate  all  terms  in  a  quadratic  model  using  fewer  experimental 
points  than  we  need  if  we  use  a  proper  fraction  of  a  2n  factorial 
array,  A  smallest  composite  design  is  obtained  when  we  have  a 
saturated  design,  i,e,,  when  we  have  exactly  as  many  points  in 
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Che  design  as  ve  have  regression  coefficients  to  be  estimated. 

To  estimate  all  terms  in  a  quadratic  response  surface  In  n 
variables  we  need  p  =  (n+l)(n+2)/2  points  to  estimate  the  p  terms 
in  the  model.  Small  designs  like  this  are  used  when  experimentation 
is  expensive,  experimental  error  Is  small,  an  Independent  estimate 
of  experimental  error  is  available,  and  a  quadratic  model  is 
adequate  to  explain  the  phenomena  under  study.  These  designs 
have  a  variance  structure  for  the  regression  coefficients  that 
is  worse  than  the  variance  structure  for  balanced  designs;  they 
give  no  estimate  of  the  experimental  error;  and  they  can  give  no 
test  of  the  adequacy  of  the  model.  For  smallest  composite  designs 
symmetric  in  the  star  points,  we  now  follow  exactly  the  same 
procedure  used  in  Section  1,2, 

A  smallest  composite  design  symmetric  in  the  star  points 
consists  of: 

1.  An  edge  point  having  the  structure 

(0  , , •  010  ...  010  ...  0) 

lxn 

1««*  i  , , ,  j  ...  n  , 

It  is  a  vector  having  l’s  in  the  i  and  j  location 
and  zeros  elsewhere.  This  point  enables  the  estimation 
of  the  x^Xj  (two  factor  interactions)  term  in  the 
quadratic  model.  We  need  one  edge  point  for  each  inter¬ 
action  term  desired  in  the  model.  If  all  interaction 
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terms  are  to  be  estimated  there  will  be  (^)  =  n(n-l)/2 
points  of  this  type. 

2.  One  center  point. 

3.  Two  star  points,  one  at  +a  and  one  at  -a,  for  each 
variable. 

Theorem  1.3:  For  a  smallest  composite  design  symmetric  in 
the  star  points  that  is  used  to  estimate  a  quadratic  response 
surface,  the  generalized  variance  decreases  with  increasing  a. 

The  method  of  proof  follows  the  same  three  steps  used  in 
Theorem  1.2, 

Step  1.  We  write  down  the  Nxn  design  matrix  which  is  "expanded" 
to  give  the  X  matrix. 


Thus 


|x»x|  *  |D  -  B'  (f1  B|  * 


2a2  2a2  ,  .  ,  2a 

0  0  ...  0 

0  0  ...  0 

•  •  • 

•  •  • 

•  •  • 

0  0  ...  0 

2a4  0  0 

0  2a4  0 

0  0  2a 


which  upon  using  Theorem  [1]  again  is 

(2a2)n(2a4)nU  +  2n  -  n4a4]  =  22n  a6n 

2a* 


which  is  an  increasing  function  of  a. 

If  S  <  (^)  interactions  are  to  be  estimated,  exactly  the 
same  determinant  is  obtained.  This  can  be  seen  by  induction. 
If  one  interaction  vector  is  removed  from  the  X  matrix,  we  can 
use  exactly  the  same  proof  and  note  that  the  determinant  does 
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not  change.  Similarly,  we  can  note  that  if  (k-1)  interaction 
vectors  are  removed,  the  value  of  the  determinant  will  not  be 
changed  by  the  removal  of  the  kc  vector.  This  procedure 
enables  us  to  remove  all  undesirable  interactions  terms  until  we 
have  left  only  the  S  interaction  terms  which  we  wish  to  estimate. 

If  the  design  is  restricted  to  a  hypercube  having  sides  of 
length  two  centered  at  the  origin,  the  above  Immediately  gives 
the  optimum  value  of  a  as  o  ■  1, 

1,4  Optimization  of  Smallest  Composite  Designs 
Bnsymmetric  in  the  Star  Points 

A  smallest  composite  design  unsymmetric  in  the  star  points 
differs  from  a  smallest  composite  design  symmetric  in  the  star 
points  in  that  the  star  points  are  placed  at  +1  and  -a  in  the 
unsymmetric  design  rather  than  at  +a  as  is  the  case  for  the 
design  symmetric  in  the  star  points. 

Theorem  1,4:  For  a  smallest  composite  design  unsymmetric  in  the 
star  points,  the  generalized  variance  decreases  with  increasing  a. 
When  the  design  is  restricted  to  a  hypercube  having  sides  of 
length  one  (which  makes  the  star  point  distance  a/(l+a)),  the  optimum 
star  point  distance  is  l/(n+l). 

The  first  part  of  the  theorem  is  proved  following  the  same 


steps  used  previously. 


w*iere  A,  and  I  are  the  same  as  they  were  in  Section  1.3, 
Using 

D  B*  ■  1 C |  |D  -  B  (f1  B| 

B  C 


we  obtain 


[D  -  B*  C**1  B] 

* 

l+2n  l-a 

1*8  «  «  * 

l-a 

14a2 

1+a2  ... 

1+a2 

l-a  1+a2 

0 

0 

l-a3 

0 

0 

1— a  0 

14a2  ... 

0 

0 

l-a3  . . . 

0 

i-*  0  0  ...  1+a2  0  0  ...  l-a3 

1+a2  l-a3  0  ...  0  1+a4  0  ...  0 


J  *+a  0  0  ...  l-a3  0  0  ...  1+ 

We  use  the  sane  breakdown  we  used  before  letting  D.  ■  l+2n 
so  C  is  a  2nx2n  matrix.  Using  Theorem  [6]  with  b  «  d  ■  f  «  0 


we  obtain 
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restricted  to  a  hypercube  having  sides  of  length  one,  the  optimum 
star  point  distance  is  l/(n+l)»  For  this  proof  we  need  an  extension 
of  a  result  due  to  Hartley  and  Ruud  [1969] ,  which  we  discuss 
in  Section  1.6,  Restricting  the  design  to  a  hypercube  having 
sides  of  lengths  one  Introduces  a  multiplier  for  the  determinant 
of  k2n^n+2^  where  k  -  l/(l+a).  Thus  the  determinant  of  the 
restricted  XfX  matrix  is 


|X'X| 


restricted 


(l-Hx)2n(lH'2) 


o2n(l+q)2n 


q2n 

<l+a)2n(n+1) 


thus 

d|X<X|  =  2na2n~1(l+a)2n(n“1)  -  2n(irtl)a+q)2n(n*1)~1  a2n  c 
d°  (lw)4n(n+l) 

=^>  (1+a)  -  (n+1)  a  ■  0 

==>  a  *  1/n 

where  0  <  a  <  “.  We  must  transform  this  to  the  proper  units. 
Since  the  length  of  the  design  1  +  a  is  restricted  to  be  one, 
the  optimal  star  point  distance  is 

q  m  1/n  m  1 
1+a  *  1+1/n  "  n+1  * 


To  obtain  the  optimum  star  point  distance  by  another  method , 
we  reprove  the  second  part  of  the  theorem  by  rescaling  the  deter¬ 
minant  before  we  differentiate  to  solve  for  the  optimum  star 
point  distance.  Let  $  be  the  star  point  distance,  then 

1  1-8 


so 


l+o  -  1/(1-B) 

k  -  (1-8) 


and 


lx  X I  restricted 

dlx'xlrestricted 

de 


(£)2n(^)2tt(1_B>2"<n+2>  -  e^u-n)2"2 

2nB2n_1(l-e)Jn  -  2n2  32n(l-6)2rl  _1  -  0 


'  =>  (1-8)  -  n8  -  0 


* 


8  «  l/(n+l) 


30 


A  third  method  of  finding  the  optimum  star  point  distance  is  to 
rescale  the  design  matrix,  then  optimize  it  following  the  three 
steps  indicated  previously. 

Step  1.  We  show  the  structure  of  the  design  matrix  X^: 

x2  x^  «  «  «  x^  Number  of  Points 

b-  [l-«  0  ...  0  1 

l-$  0  1-B  .  .  .  0 

0  1—3  1-3  *  «  «  0  /O* 

.  .  .  .  V 

...  . 

...  . 

0  0  0  *  •  •  1-3 

1-3  0  0  ...  0 

0  1-3  0  ...  0 

0  0  1-3  ...  0  n 

•  •  •  • 

•  •  •  • 

•  •  •  • 

0  0  0  ...  1-3 

-3  0  0  ...  0 

0  -3  0  ...  0 

0  0  -3  ...  0  n 

•  •  •  • 

«  «  «  • 

•  •  •  • 

0  0  0  ...  -3 

.  0  0  0  ...  0  1 


Step  2.  We  indicate  the  structure  of  the  X*X  matrix  by  shoving  it  for  selected  rows  and  columns: 
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1.5  Optimization  of  Composite  Designs 
Unsymmetric  in  the  Star  Points 

A  composite  design  unsymmetric  in  the  star  points  consists 
of  a  proper  fraction  of  a  2n  factorial  array,  and  n  atar  points. 

Theorem  1.5:  For  a  composite  design  unsymmetric  in  the  star 
points  that  is  used  to  estimate  a  quadratic  model  the  generalized 
variance  decreases  with  a,  the  star  point  distance.  When  the 
design  is  restricted  to  a  hypercube  having  sides  of  length  one, 
the  optimal  star  point  distance  is  l/(n*f*l). 

Thus  even  though  there  are  many  more  points  in  the  cube 
part  of  the  design,  the  optimal  star  point  distance  is  the  same 
as  that  obtained  for  smallest  composite  designs  unsymmetric  in 
the  star  points.  The  proof  will  follow  the  same  three  steps 
used  in  the  previous  proofs  though  the  algebra  is  more  involved 

and  more  involved  matrix  results  are  used, 

* 

To  prove  the  theorem,  we  reduce  the  determinant  to  a  constant 
(independent  of  o)  times  the  result  for  Theorem  1.4. 

Possible  composite  designs  unsymmetric  in  the  star  points 
for  estimating  a  quadratic  model  for  n  <_  11  are  listed  in 
Table  1,2, 
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Table  1,2 

Common  Unsymmetric  Composite  Designs 


Variables 

Terms  In 
Quadratic  Model 

Points  in 
Design 

I .actional 
Factorial 

2 

6 

6 

22  =•  4 

3 

10 

11 

23  =  8 

4 

15 

20 

24  -  16 

5 

21 

37 

25  -  32 

5 

21 

21 

25”1  «=  16 

6 

28 

70 

26  «  64 

6 

28 

38 

26"1  -  32 

7 

36 

135 

27  «  128 

7 

36 

71 

2 7-1  =  64 

7 

36 

39 

27”2  =  32 

8 

45 

264 

28  -  256 

8 

45 

136 

28-1  *  128 

8 

45 

72 

28”2  «  64 

9 

55 

521 

9 

2  -  512 

9 

55 

265 

29-1  «  256 

9 

55 

137 

29~2  -  128 

9 

55 

73 

29“3  »  64 

10 

66 

1034 

210  -  1024 

2i°-i 


10 


66 


522 


*  512 
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Table  1,2  (Continued) 

Common  Unsymmetric  Composite  Designs 


Variables 


Terms  in 
Quadratic  Model 


Points  in 
Design 


Fractional 

Factorial 


10 

66 

266 

210"*2  -  256 

10 

66 

138 

210“3  =  128 

10 

66 

74 

210"4  -  64  * 

11 

78 

2059 

211  =  2048 

11 

78 

1035 

2li~1  *  1024 

11 

78 

523 

211”2  -  512 

11 

78 

267 

211"3  =  256 

11 

78 

139 

211”4  *  128 

While  the  designs  indicated  with  a  *  have  more  experimental 
points  than  there  ere  parameters  in  a  full  quadratic  model,  the 
alias  structure  is  such  that  all  parameters  in  a  quadratic  response 
surface  cannot  be  estimated.  See  Section  1,6,2, 


where 


For  the  six  interactions  obtained  with  four  variables,  the 
pattern  of  B  is: 
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12 

13 

14 

23 

24 

34 

12 

" 2n‘k-2 

2n-k-3 

2n"k-3 

2n-k-3 

2n-k-3 

2n-k-4 

13 

2n-k-3 

2n-k-2 

2n-k«3 

2n-k-3 

2n-k-4 

2n-k-3 

14 

2n-k-3 

2n-k-3 

2n-k-2 

2n-k-4 

2n-k-3 

2n-k-3 

23 

2n-k-3 

2n-k-3 

2n-k-4 

2n-k~2 

2n~k-3 

2n-k-3 

24 

2n~k-3 

2n-k~4 

2n-k-3 

2n-k-3 

2n~k~2 

2n~k-3 

34 

2n-k-4 

2n-k-3 

2n-k-3 

2n-k-3 

2n-k-3 

2n-k-2 

Simplifying,  we  obtain 


«T» 

M 

A' 

«r» 

A » 

-  |S| 

M-  A*  [B]-1  [JAA] 

A » 

J*A  A 

B 

where  M(2n+l)x(2n+l)  is  a  matrlx  whose  elements  are  a  function 
of  o,  5  does  not  contain  a  and 


J' 
A  * 
A' 


IP)'1  UAA] 
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C1  c2^1  c2^t 
C2*I  c^I  4*  c4JJ»  c^I  4*  c^JJ’ 

c?J  c3I  +  c^JJ*  c3I  +  c4JJ* 

(2n+l)  x  (2n+l) 

where  c^,  c^*  cj»  are  constants  independent  of  a.  Thus, 

the  |x*x|  reduces  to 

r  (s-a)J'  <s+a2)J» 

(s-a)J  (ct2-Hn)I+qJJ»  (-a3-Hn)I+qJJ« 

(s+a2)J  (-«3-Hn)I+qJJ*  (a4-Hn)I+qJJ* 

(2n+l)  x(2n+l) 

Use 

D  B*  . 

-  |d|  |C  -  B  D"1  B*| 

B  C 
to  reduce  |x'x|  to: 

Kj  al  +  bJJ*  cl  +  dJJ’ 

cl  +  dJJ’  dl  -  fJJ* 

2n  x  2n 
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1,6  Belated  Mathematical  Results 


1.6.1  Effects  of  linear  transformations 

In  the  matrix  if  any  column  is  multiplied  by  a  constant 

2 

k  then  the  determinant  will  be  multiplied  by  k  ,  This  is  easily 

seen  as  the  modified  X  matrix  can  be  written  as  X__  P  „ 

Hxp  pxp 

where 


P  - 


I 

0 

0 


0 

k 

0 


0 

0 

I 


and 


|P’X«X  p(  «  |p*| |x*x|  |p|  -  k2  |x‘x|  . 


i 

j 

a 

1 

'■3 

% 

a 

-! 

3 


Multiplying  a  column  in  the  design  matrix  X^  by  k  causes  the 

following  changes  in  the  X  matrix.  The  linear  term  is  multiplied 

2 

by  k,  the  quadratic  term  is  multiplied  by  k  and  (n-1)  interaction 
terms  (for  a  full  quadratic  model)  are  multiplied  by  k;  so  this 
multiplies  the  determinant  by  k2^n+2\  A  result  due  to  Hartley 
and  Ruud  [1969]  is  Theorem  1.6,1. 

Theorem  1.6.1:  If  the  n  vectors  X^,  are  transformed  to  vectors 
Ztby 


1 


mt'MHMBlMttM  teaiUMSuUlUSI ! 


mmm 
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\  «  xt  +  $ 


where  Xt  is  a  row  vector  in  the  design  matrix  or  a  row  vector 
in  the  X  matrix  with  6^  -  0,  then 

|x*x|  «  |z»z|  . 


Proof:  Z  can  be  written  as 


where 


so 


^N*p  *  ^Nxp  ^pxp 


1 

0 


|a|  »  l 


thus 


z*z|  -  |a*|  |x»x|  |a|  -  |x*x|  . 
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By  combining  the  aboye  results  we  can  see  how  the  determinant  is 

changed  by  the  usual  linear  transformations*  For  a  particular 

example  if  each  element  of  the  design  matrix  for  a  quadratic 

model,  is  transformed  by  Z. .  ■  kx..  +  6.  then  the  determinant 

3 

becomes 

k2n(n+2)  |x,x|  # 

1*6*2  The  estimable  terms  in  the  model 

Hartley  [1959]  discusses  the  terms  in  a  model  that  can  be 
estimated  using  a  composite  design.  We  will  extend  his  result 
slightly  using  estimabllity  considerations.  We  define  estimability, 
give  a  standard  theorem  on  estimability,  then  extend  Hartley’s 
result. 

Definition:  A  linear  function  X’B  is  said  to  be  linearly 
estimable 3  a  vector  a^x^  such  that  E(a*y)  ■  X'B, 

Theorem  1,6,2:  X’f5  is  estimable  <r*>  X*  ■  a’X. 

Proof :  ■£> 

X’g  is  estimable  ^  3a»  E(a’y)  »  X*f3  but 
E(a'y)  *  aX0  X’  «  aX. 

<T 

a’X  •  X  is  a  constant  set  of  equations*  Let  a'  be  any 
solution,  then 


E(a’y)  -  a'XB  -  X’B 
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Theorem  1.6.3: 

(a)  For  a  composite  design  symmetric  in  the  star  points 
it  is  always  possible  to  estimate  the  following 
coefficients  of  a  quadratic  response  surface: 

The  constant  term,  all  linear  coefficients, 
all  quadratic  coefficients,  one  of  the 
product  coefficients  (interactions)  selected 
from  each  of  the  alias  sets. 

(b)  It  is  not  possible  to  estimate  more  than  one  of  the 
product  coefficients  from  each  alias  set. 

This  theorem  is  an  extension  of  Hartley’s  result  in  two 

ways; 

(1)  We  allow  main  effects  to  be  used  in  the  defining  contrast 

V  n 

in  the  (1/2  )2  fracticnai  replicate,  and 

(2)  We  consider  there  to  be  2n~^  alias  sets  by  allowing 
the  defining  contrast  to  be  counted  as  an  alias  set. 
Hartley  allowed  2n“^  -  1  alias  sets. 

Two  simple  examples  using  this  extension  are: 

(1)  A  two  variable  composite  design  having  seven  points. 

2-1 

This  design  uses  a  2  fractional  factorial  with  alias 


structure  I  ■  AB 
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(2)  A  three  variable  design  having  nine  points  with  a 
3-2 

2  fractional  factorial  with  alias  structure 
I  =  A  “  BC  =  ABC.  This  design  allows  the  estimation 
of  the  constant  term,  three  linear  terms,  three  quadratic 
terms,  and  two  of  the  three  interaction  terms,  the  BC 
interaction  and  either  the  AB  or  the  AC  interaction. 

The  experimenter  must  be  careful  when  he  is  using  highly 
fractionated  designs.  From  each  alias  set  one  interaction  term 
can  be  estimated,  the  choice  of  which  interaction  term  to  estimate 
is  that  of  the  experimenter.  This  choice  can  change  the  interpreta¬ 
tion  of  the  experiment. 

Proof  of  Theorem  1.6.3. 

From  the  center  point  and  2n  star  points  we  can  find  a 

vector  such  that  E(afy)  ■  8^  for  the  constant  term,  n  linear 

n-k 

terms,  and  n  quadratic  terms,  2n+l  terms  in  all.  From  the  2 
fractional  factorial  we  can  find  2  estimable  functions;  the 
Yates  algorithm  will  give  us  the  estimable  functions  we  desire. 

These  functions  will  contain  some  combination  of  the  following: 

The  constant  term,  main  effects,  two  factor  interactions, 
higher  order  interactions  (which  are  not  used  in  a 
quadratic  model) . 

Since,  if  A. 8  is  estimable  E  A  8  is  estimable,  we  can  combine 

i 

the  A’s  from  the  star  with  the  A’s  from  the  fractional  factorial 
to  estimate  all  terms  in  a  model  that  picks  only  one  interaction 
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term  from  each  of  the  2  estimable  functions  of  the  fractional 
factorial.  Note  that  2n  ^  may  be  greater  or  less  than  (^) 
the  number  of  interactions  so  we  may  have  a  saturated  design  or 
have  many  terms  left  over  for  estimating  error.  This  completes 
part  (a)  of  the  theorem. 

After  noting  that  the  2n+l  star  and  center  points  make  no 
contributions  to  the  estimation  of  the  interaction  terms  (for 
these  points  the  row  vector  of  the  design  matrix  X^  has  at  most 
one  non-zero  element,  thus  for  these  rows  all  interaction  elements 
of  the  X  matrix  are  zero) ,  we  can  Immediately  see  that  it  is 
Impossible  to  estimate  more  than  one  interaction  term  from  each 
of  the  alias  sets.  This  completes  the  proof  of  the  theorem. 

For  composite  designs  unsymmetric  in  the  star  points  a  very 
similar  theorems  using  the  same  method  of  proof  is: 

Theorem  1.6.4: 

(a)  For  a  composite  design  unsymmetric  in  the  star  points 
it  is  always  possible  to  estimate  the  following 
coefficients  of  a  quadratic  response  surface: 

All  quadratic  terms  and  one  coefficient  from 
each  of  the  2  alias  sets. 

(b)  It  is  not  possible  to  estimate  more  than  one  of  the 
product  coefficients  from  each  of  the  alias  sets. 
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A  particular  difference  between  composite  designs  symmetric 
in  the  star  points  and  those  unaymmetric  in  the  star  points  is 
that  an  alias  structure  having  rrords  with  three  characters, 
e.g.,  I  =  ABC,  allows  the  estimation  of  all  terms  in  a  quadratic 
response  surface  for  the  former  design  but  not  for  the  latter 
design.  For  the  latter  design  note  that  if  you  use  the 
principle  block  of  the  I  *  ABC  alias  structure  the  N  design 
points  in  n  dimensions  project  onto  seven  distinct  points  in 
three  dimensions.  With  this  alias  structure  the  experimenter 
would  commonly  choose  to  estimate: 

(a)  The  constant  term,  three  linear  terms,  and  three 
quadratic  terms,  or 

(b)  The  constant  term,  three  linear  terms,  and  three 
interaction  terms. 

In  case  (b)  you  would  be  using  the  star  points  to  estimate 
linear  terms  and  the  factorial  structure  to  estimate  the  constant 
terms  and  the  Interaction  terms. 

In  general  such  an  alias  structure  would  not  be  recommended, 
rather  alias  structures  which  have  at  least  five  characters  for 
each  word  in  the  alias  set,  i.e.,  I  =  ABCDE,  which  allow  the  estima¬ 
tion  of  all  terms  in  a  quadratic  response  surface  would  be  used. 


^eg 
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1,6,3  Smallest  fractional  factorials  -  disproof  of  a  conjecture 

Using  the  alias  structure  I  «  ABC  it  is  easy  to  construct 
composite  designs  symmetric  in  the  star  points  for  estimating  all 

n_k 

terms  in  a  quadratic  response  surface.  These  designs  use  a  2 
fractional  factorial  with  k  £  n/3.  It  has  been  conjectured  that 
k  must  be  £  n/3  for  all  n,  if  all  terms  in  a  quadratic  response 
surface  are  to  be  estimated.  By  enumeration  of  the  possible  alias 
structures  it  can  be  seen  that  the  conjecture  is  true  for  n  10. 
These  are  the  designs  which  are  commonly  used  in  practice.  The 
conjecture  is  not  true  for  all  n,  however.  We  disprove  this 
conjecture  by  listing  two  different  alias  structures  for  n  *  11, 
k  =  4  that  enable  the  estimation  of  all  terms  in  a  quadratic 
reeponse  surface.  The  first  design  has  one  three  character  alias 
word  in  the  alias  set,  the  second  example  has  five  or  more 
characters  for  all  words  in  the  alios  set.  The  second  example 
<?an  be  found  in  Box  and  Hunter  [1961] ,  Ws  have  underlined 
the  words  which  we  use  to  generate  the  alias  structure. 

Example  1. 

1  "  ABODE  «  ABFGH  »  CDFGI  -  ADEFHIJK  n  CDEFSK  *  ABEFGI  -  ABCDHI  « 
EHI  »  BCFHIJK  ■=  BDEGIJK  «  ACGIJK  *  ACEGHJK  -  BDGHJK  -  BCEFJK  * 


ADFJK 
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Example  2. 

I  *  ABCGH  -  BCDEI  -  ACDFJ  -  ABCDEFGK  -  ADEGHI  -  BDFGHJ  -  ABEFIJ  = 
CEFGHIJ  -  DEFHK  =  AFGIK  *  BEGJK  -  BCFHIK  -  ACEHJK  =  CDGIJK  » 

ABDHIJK 

1.6.4  Optimum  design-optimum  composite  design  comparison 

Figure  1.6.1  compares  the  optimal  six  point  design  obtained 
by  Ruud  [1969]  with  the  optimal  six  point  composite  design.  Both 
designs  are  restricteu  to  a  square  with  sides  four  units  long. 
Ruud's  design  for  estimating  a  quadratic  response  surface  has 

5 

175.  x  10  as  the  value  of  the  determinant  of  the  X'X  matrix  while 

the  composite  design  has  20.7  x  10^.  The  ratio  of  these  two  values 

th 

is  8.45.  A  better  comparison,  perhaps,  is  the  2  x  p  root  of 

this  ratio  which  is  a  measure  of  the  ratio  of  the  average  standard 

deviation  of  a  coefficient  in  the  less  efficient  design  to  the 

standard  deviation  of  a  coefficient  in  the  more  efficient  design. 

1/12 

In  this  example  with  p  -  6, (8.45)  ■  1.195.  This  increase  in 

standard  deviation  reflects  the  penality  paid  in  less  precise 
estimates,  for  using  the  more  easily  obtained  composite  design 
rather  than  the  optimal  design  obtained  by  solving  a  non-linear, 
non-convex  program. 

N”te  that  the  optimum  composite  design  requires  only  three 
levels  for  each  factor,  while  the  Ruud  design  requires  four  levels 
for  each  factor.  In  fact  we  can  observe  that  all  optimum  composite 


Figure  1,1 


Six  Point  Design  Comparison 


|X’x[  -  175.  x  10' 


Ruud' 8  Six  Point  Design 


|x’x| 


■  20.7  x  10s 


Optimal  Composite  Design 


designs  on  a  hypercube  require  only  three  levels  for  each  factor. 

This  is  extremely  gratifying  as  in  industrial  experimentation 

the  implementation  of  a  factor  level  is  often  costly.  Economic 

considerations  might  therefore  indicate  that  an  optimal  composite 

design  should  be  used  rather  than  a  more  precise  optimal  design 

that  requires  more  levels  for  each  factor. 

The  use  of  an  optimal  composite  design  does  not  necessarily 

incur  any  penality  at  all.  For  example  a  nine  point  optimal  design 

on  a  square  is  the  same  as  a  nine  point  composite  design  symmetric 

in  the  star  points.  These  are  the  same  as  Kennard  and  Stone's 

[1969]  and  Box  and  Draper's  [1971]  optimal  nine  point  design.  All 

2 

the  above  designs  are  3  factorial  designs  having  determinant 
340  x  106  when  the  sides  of  the  square  have  length  four. 
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APPENDIX 

Some  useful  results  about  Matrices  and  Determinants: 
Theorem  [1] 


“  |A22|  |A11  "  ^2^22^21 1 

m*n 


when  A^2  Is  square  and  non-singular. 
Proof:  Theorem  1.50,  Grayblll  [1961]. 


^1 

^2 

A21 

A22 

Theorem  f 2] 


(I  +  AB)**1  -  (I  -  A(I+BA)~lB) 


Proof:  Multiply  out  to  verify. 


Theorem  f3l 


I  +  AB[  -  |l  +  BA | 


Proof :  Consider 


xj  8 


and  use  Theorem  [1]  twice  to  obtain 


h  B 

-A  I. 


h  rj  +  bik1a 


h  + 


Theorem  f 4 1 

|al  +  bJJ'ln)<n  -  a""1  (a  +  nb) 


Proof: 


|al  +  bJJ*|  -  an  |i  +  £jj»| 

“  an  U  +pJ’jj  by  Theorem  [3J 


an(l  +  — ) 
a 


a""1 (a  +  nb) 


Theorem  [51 


[al  +  bJJ']"^  «  [cl  +  dJJ»] 

n*n  Jn*n 


where  c  =  —  and  d 
a 


b 

a(a+nb)  * 


Proof:  Multiply  out 
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Theorem  f61 

al  +  bJJ» 
cl  +  dJJ’ 

Proof: 

al  +  bJJ* 
cl  +  dJJ* 


cl  +  dJJ’ 


el  +  fjj* 


2nx2n 


(ae-c2)11”1  [(a+nb)(e+nf) 


(c+nd)2] 


cl  +  dJJ» 
el  +  fjj* 


an  ^"(a+nb) 


[el  +  fjj’]  -  [cl  +  dJJ’] 


an-1(a+nb) 


nbcd 

a(a+nb) 


h#) 


i+4f - 


2cd  bc^ 


a  a(a+nb) 


n  +  ^cd  4.  nbd2  ) 

[a  a(a+nb)  a£a+nb)  J 


Jjj' 


r  2' 

-  Cl 

H 

1 

e 

1 

“1 

1 

e  -  —  +  nf  - 
a  a 


nd  ,  nb  ,2  29 

"  ~  +  a(a+nb)  (c  +  2ncd  + 


(ae-c2)*1”1  { (a+nb) (e+nf )  -  (c+nd)2>  # 


